Decentralized and Model-Free Federated Learning: Consensus-Based Distillation in Function Space

This paper proposes a fully decentralized federated learning (FL) scheme for Internet of Everything (IoE) devices that are connected via multi-hop networks. Because FL algorithms hardly converge the parameters machine (ML) models, this focuses on convergence ML models in function spaces. Considering representative loss functions tasks e.g, mean squared error (MSE) and Kullback-Leibler (KL) divergence, convex functionals, directly update spaces could to optimal solution. The key concept is tailor consensus-based optimization algorithm work space achieve global optimum distributed manner. first analyzes proposed space, which referred as meta-algorithm, shows spectral graph theory can be applied manner similar numerical vectors. Then, distillation (CMFD) developed neural network (NN) implement meta-algorithm. CMFD leverages knowledge realize aggregation among adjacent without parameter averaging. An advantage it works even with different NN learners. Although does not perfectly reflect behavior discussion meta-algorithm's property promotes an intuitive understanding CMFD, simulation evaluations show using several tasks. results also achieves higher accuracy than weakly networks, more stable methods.